Distribution of training labels

Pixel intensity representation of the training set

Training data in 2D

We adopt t-distributed Stochastic Neighbor Embedding(t-SNE) to represent the data in 2D.(see details )

Neural networks

We use feedforward neural network which has the following structure

• input layer: consists of neurons that receives the input data each neuron in the input layer represents a feature of the input data
• hidden layer: one or more hidden layers placed between the input and output layers, responsible for capturing complex patterns
• output layer: final output of the network. Here, the number of neurons represents the number of digits

NN with regularization

Specific NN models considered

• NN with dropout regularization
• NN with ridge regularization
• NN with lasso regularization

Multinomial logistic regression

Multinomial logistic regression is equivalent to a NN with just input and output layers. There are no hidden layers.

• Output layer with softmax

To be edited

Let’s train the network for 30 epochs, using a default batch size of 32. This means that images from the training set will be presented to the network in batches of 32 at a time, and for each batch, the SGD algorithm will update the network’s weights by an appropriate amount. Then another batch of 32 images will be presented, and so on, until all 60,000 training images in the dataset have been processed, which constitutes one epoch of training. This entire cycle will be repeated for 30 epochs. As training proceeds, the network’s error (loss) on both the training and testing/validation sets will be shown on the left graph, and the accuracy on each of these sets will be shown on the right graph. The accuracy is simply the fraction of input images that the network classifies correctly. A classification is considered correct if the largest output value on the output layer corresponds to the target classification

TDA Workflow